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HyperVideo Tracking and Reporting System 



Field of the Invention 

The invention relates generally to a hypervideo system incorporating hot links 
within a video presentation, and more specifically to a hypervideo tracking and 
reporting system. 



This application hereby incorporates by reference related pending U.S. patent 
appHcations 09/200,146, filed 11/25/1998 , titled "Streaming Hypervideo and 
Dynamic Hypervideo"; and 08/882,512, filed 06/25/97, titled "System and Method for 
Linking Information to and Accessing Information From a Video". 

Claim of Priority 

This application claims priority from U.S. provisional application 60/167,165, 
filed 11/23/99, titled "Hypervideo Suite". 



As personal computers and access to the Intemet have become increasingly 
common, computers have become more widely relied upon for communication. 
Businesses provide product informafion, customer support, and ordering capability via 
the Intemet, e-mail is widely used for personal and business communication via the 
Intemet in place of telephone or postal mail, and discussions relating to a wide variety 



Incorporation of Related Matters 



Background of the Invention 




of specific topics are conducted in Internet-based discussion groups. As technology 
and access to the Internet continue to increase, it is reasonable to expect that 
communication via the Internet will continue to increase in popularity and the methods 
in which the Internet is used to communicate will continue to expand. 
5 Web pages 6P ^vebsites comprising a part of the World Wide Web (WWW) are 

perhaps the most common means of posting information for mass viewing or for 
providing services such as e-commerce ability to the public at large. Web pages are 
defined via a markup language that can be transmitted via the Internet to website users, 
who run web browser applications that interpret the markup language information and 

10 render web pages on the user computer systems. Markup languages such as HTML 
allow presentation of text, graphics, menus, tables, and other useful structures to users 
via the Internet connections that link web page servers and user client systems running 
web browser applications. One common feature incorporated into web pages is known 
as a "Hyperlink", which is typically text or a graphic representing other content, that 

15 when clicked on using a mouse cursor or otherwise selected loads and presents the 
represented other content. 

But, as bandwidth and computational power in computerized systems continue 
to increase, the types of information presented via the World Wide Web and other 
sources continues to advance. Now, it is not uncommon to find relatively large and 

20 complex streaming audio or video broadcasts of content in addition to more traditional 
text and still image content, and such content continues to become more common. 
However, content such as video has typically been presented as a stand-alone 
presentation, without incorporating links to other content. 
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The concept of Hypervideo seeks to provide this hyperUnk-type functionahty 
to video presentations, enabUng Unking and retrieval of information related to objects 
or images presented in a video presentation. For example, a user watching a football 
game may move a cursor over a player and cUck on the player's image to see statistics 
5 on that player, or may similarly select a product in a commercial or television show for 
more information on the product. 

Hypervideo capability is currently provided via a proprietary hypervideo 
software system called Hypervideo Suite. This software provides a multimedia 
designer the tools to create video with hyperlinks, such as may be viewable over the 

10 Intemet or other broadband network using a hypervideo player or other media viewer. 
Incorporation of hyperlinks into video using Hypervideo Suite technology is hoped to 
become the preferred method of providing hyperlinks in video content as broadband 
network access becomes commonplace. 

But, along with tools to facilitate providing hypervideo content, it is desirable 

15 to track the use of such content. Because such content remains relatively expensive to 
produce and distribute, evaluation of commercial effectiveness of hypervideo content 
for various applications is key to cost-effective or profitable use of hypevideo. 
Tracking use of content can serve a variety of other fiinctions, such as tracking 
progress of training or educational material. While web servers often provide statistics 

20 on the number of times selected elements have been downloaded and other such 
information, they do not track a user's progress through a series of links where the 
links include clicking through hotspots or links in video, or provide other information 
key to understanding how content is used. 
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What is needed is a system that is capable of tracking and reporting user 
behavior in a Hypervideo system. 

Summary of the Invention 

5 The present invention facihtates tracking and reporting of user behavior in a 

hypervideo system. The system provides tracking and recording of media viewed by a 
user, where media comprises hypervideo video-enhanced pages and other media types. 
The system is operable to create a user-configurable report, comprising data regarding 
user use of the media, such as indicating the number of times each media has been 
10 viewed and indicating the number of times each path has been taken through each 
hypervideo appUcation. 

Brief Description of the Figures 

Figure 1 is a computer screen image of a hypervideo appUcation report, 
1 5 consistent with an embodiment of the present invention. 

Figure 2 is a table illustrating application executions for each application 
sorted by day of the week the execution occurred, consistent with an embodiment of 
the present invention. 

Figure 3 illustrates a computer screen image that represents a video-enhanced 
20 page report generated consistent with an embodiment of the present invention. 

Figure 4 illustrates a report indicating the media path taken through each 
application, consistent with an embodiment of the present invention. 

Figure 5 shows a path detail report screen image, consistent with an 
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embodiment of the present invention. 

Figure 6 is a screen image of a hotspot report ^consistent with an embodiment 
of the present invention. 

Figure 7 is a screen image of an advertising media hotspot report,consistent 
with an embodiment of the present invention. 

Detailed Description 

In the following detailed description of sample embodiments of the invention, 
reference is made to the accompanying drawings which form a part hereof, and in 
which is shown by way of illustration specific sample embodiments in which the 
invention may be practiced. These embodiments are described in sufficient detail to 
enable those skilled in the art to practice the invention, and it is to be understood that 
other embodiments may be utilized and that logical, mechanical, electrical, and other 
changes may be made without departing from the spirit or scope of the present 
invention. The following detailed description is, therefore, not to be taken in a limiting 
sense, and the scope of the invention is defined only by the appended claims. 

The present invention provides in various embodiments a system facilitating 
tracking and reporting of user actions in a hypervideo system. The system provides 
tracking and recording media viewed by a user, where media comprises hypervideo 
video-enhanced pages (VEPs) and other media types. The system is operable to create 
a user-configurable report, indicating data relating to user access of the media such as 
the number of times each media has been viewed and the number of times each path 
has been taken through each hypervideo application. 




Hypervideo is a system enabling linking information to and accessing 
infomiation from a video. Hypervideo contains one or more regions of interest, or 
hotspots, where each hotspot is linked to one or more targets. Hypervideo permits a 
user to interact with video, by selecting hotspots to retrieve selected information or 
5 perform other actions associated with the image the hotspot represents. The user can 
navigate hypervideo from any base such as HTML, an image, video, or other media 
types, and may similarly retrieve from within hypervideo any type of media. The 
media targets viewed from within hypervideo may be viewed or executed serially, 
simultaneously, or in any other manner. 

10 An authoring tool is used to create hotspots within video to create hypervideo 

apppHcations by defining regions of interest and associating those regions with other 
media content. The hotspot similarly may be defined in a picture, such as a bitmap 
graphic image. The hotspot may then be selected by a user, such as by placing a 
mouse cursor over the hotspot and clicking the mouse. Upon activation, the linked 

1 5 media associated with the hotspot is viewed or executed. Multiple media may be 
linked to a hotspot in some embodiments of the invention, so that selecting or 
actuating a hotspot retrieves or executes one or more different linked media. 

Hypervideo media hotspots and media content may be initiated via user 
selection such as by mouse click, may be actuated via automatic activation such as an 

20 API call in a Javascript program, or in any other way in various embodiments of 
hypervideo systems. 

Hypervideo in one embodiment of the invention will be provided to a user via 
a server providing content over a broadband network, and the users of hypervideo will 




view hypervideo content using hypervideo content viewer applications or h>pervideo 
viewer plugin applications within standard web browsers. Other embodiments may 
include delivery of hypervideo content via machine-readable mediums such as CDs, or 
via any other method of delivering content to a user. The hypervideo content may also 
5 be viewed in some embodiments using standard viewers, rather than dedicated or 
proprietary hypervideo viewers. 

Hypervideo will be useful in a variety of applications, such as interactive 
television, games, toruism information, and home shopping. It is anticipated that due 
to the flexibility of the hypervideo system, a variety of other applications exist and 
10 will be developed, all of which are consistent with and within the scope of the present 
invention. 

A hypervideo apphcation provides a tree of media that includes video- 
enhanced pages, where video-enhanced pages are pages of content that are displayed 
or otherwise presented to a user. For example, video, images, audio, or HTML pages 

1 5 may be video-enhanced pages, if they are pages that form a part of the sequence of 
content displayed by the hypervideo apphcation. The video-enhanced page (VEP) is 
further defined as the hypervideo application executes as only that content that is 
currently being displayed in the front or current level of the application. In some 
embodiments of the invention, when more than one element of content or media are 

20 displayed at a time, the VEP is the content type that has the highest priority or that is 
displayed as a larger representation. Content types are ordered in one such 
embodiment so that video is the highest priority, followed sequentially by images, 
audio, and HTML pages. 
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Tracking and reporting of hypervideo content is acomplished via a novel 
tracking and reporting system. When a user requests a hypervideo VEP or media, the 
request is logged and the time the VEP is viewed is recorded. The logged information 
may be automatically analyzed to create performance statistics, to alter the operation 
5 of the server, or to perform other functions, but is also available to a reporting tool that 
is operable to peform functions such as reporting and summarizing information about 
the hypervideo media. 

One example of a hypervideo application report is shown in Figure 1 . The 
screen image of Figure 1 shows the applications listed at 101, and shows for each 

10 application displayed the number of user sessions during which the application was 
executed at 102. At 103, the number of unique users to have viewed each appHcation 
is displayed, and the average time each user spends within each application is shown at 
104. In some further embodiments of the invention, some or all of this information is 
presented graphically, as is shown at 105. 

15 This information is also viewable sorted by time of day, day of the week, or 

other criteria as is shown in Figure 2. Figure 2 shows generally a table illustrating an 
example embodiment of the present invention in which application executions are 
listed by day of the week, which may allow a hypervideo system provider to better 
understand demand and use of the hypervideo media content provided. 

20 Figure 3 illustrates a computer screen image that represents a video-enhanced 

page (VEP) report generated consistent with the present invention. Each video 
enhanced page listed in the table is represented both by an icon indicating the media 
type at 301, and a name at 302. Ideally, the name will be descriptive of the content of 
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the media, to facilitate user interpretation of the report. At 303, the number of views 
of each media (including VEP) Hsted in the report is shown, and the number of user 
sessions during which each media was viewed is listed at 304. At 305, the report table 
indicates the number of unique users to have viewed each media, and the average time 
5 that users view each listed media is shown at 306. Media other than VEP media are 
included in the embodiment of the invention shown in Figure 3 for a variety of 
reasons, including representation of media viewed while not in the front or current 
position as is required in some embodiments for media to be considered a VEP. 

The user may configure the report tool in some embodiments of the invention 

10 to report data for only selected types of media, to display statistics for only certain 

dates, or to otherwise select the type of data presented in a report. For example, a user 
may wish to view data starting at the first day of the current month for VEP media 
only, and may create a custom report including only such data. 

In a further embodiment of the invention, the number of times each media has 

15 been viewed as shown at 303 of Figure 3 does not include returning to a media in a 
single session where the media path brings a user back to a media. In this manner, 
double-viewing of media that causes a user to view other content and then return does 
not skew the statistics presented in the report. 

The report may also be limited in some embodiments to showing statistics for 

20 only a selected number of top or bottom-ranked applications. For example, a report 
may be limited to the most frequently viewed 20 applications on a server. This 
flexibility allows the user to further customize reports and derive additional 
information from the reporting tool. 




For looping media such as video or audio, the number of total views and the 
number of distinct views not resulting from looping may be indicated, so that the 
number of views reported is not distorted by the looping process. For example, a 
video viewed 120 times total but only 40 times in distinct views not a result of looping 
5 indicates that the video looped three times on average each time it was played. 

The statistics screen may further indicate in select embodiments of the 
invention the reason for the media start or presentation. For example, an automatic 
start from an application, a hotframe, or an end-of-media triggered start may be 
reported individually or as automatic starts, and hotlinks, API calls, and hotspot 
10 activations may be reported separately or as user-triggered starts. This information 
may be summarized or otherwise characterized such as by icon in fiirther 
embodiments of the invention. 

Figure 4 illustrates a report indicating the media path taken through each 
application, listed in order of the total number of user executions of each application 
1 5 following each path displayed. A path is defined in the embodiment of the invention 
illustrated in Figure 4 as the page flow of the application from start to end of a user 
session, including VEPs and other media. The number of user sessions per application 
path is shown at 401, and the number of unique users following each application path 
is shown at 402. At 403, the percentage of total application executions that took each 
20 path is listed, indicating which paths through the application are most common. The 
average amount of total time spent taking each path is listed at 405. 

The report shown in Figure 4 is in some embodiments user-configurable to 
include or present data in a way the user desires. For example, a minimum path length 
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may be defined by the user, so that paths taken through an application not meeting or 
exceeding the minimum number of media or VEPs viewed will not be displayed in the 
report. Also, paths that are not entirely identical may be grouped together for 
reporting purposes if the have selected path segments in common. In such an 
5 embodiment, a user may choose to group all paths that have the first five elements in 
common, or may choose to group paths that have some other selected segment in 
common, so that they will be aggregated and single path statistics will be displayed for 
each group. 

As is shown at 405, the path taken through an application may be represented 
10 by icons representing each media or VEP in the path. In alternate embodiments, 

media names, icons, or other methods may be employed to uniquely identify each path 
within the report. Icons may be used in some embodiments of the invention to 
visually represent additional data, such as what triggered a hotspot activation, different 
media types, or other data. 
15 A path report is provided in some embodiments of the invention, as is shown at 

Figure 5. The path report shown in Figure 5 shows only VEPs, but the invention in 
alternate embodiments also includes other media. The report shows at 501 a list of 
media contained in a currently-displayed VEP, and the hotspots on each media. Data 
regarding previous VEPs which led to the current VEP is shown generally at 520, and 
20 data relating to the VEPs selecte from the current VEP is shown generally at 530. The 
Unks from the various hotspots in the currently-displayed VEP or media to other VEP 
or media 503 are shown at 502, along with various statistics regarding the frequency 
and time users progress from the current media to each of the next linked media. The 
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average time a user spends after progressing to each of the next media is shown in 
table form at 504, and the total number of user sessions or application executions in 
which a user proceeds to each media is listed at 505. At 506, the percentage of total 
media selections attributed to each media is shown, indicating which paths are most 
5 likely to be taken. 

This path detain report therefore shows both how users arrived at the current 
VEP and the paths users took from the current VEP, ndicating paths that are effective 
and desira 

Figure 6 is a table that shows a detailed hotspot report, showing statistics for 
10 the most frequently selected hotspots in an application. In other embodiments of the 
invention, a hotspot report may comprise statistics for hotspots meeting certain use or 
date criteria, or other criteria selected by a user. The report shows statistics reflecting 
the use of each hotspot, including each hotspot name at 601, type at 602, and location 
at 603. The number of sessions or application executions in which each hotspot is 
15 viewed is shown at 604, and the number of movements to each hotspot via use of a 
mouse (whether the hotspot was actuated or not) is shown at 605. The number of 
times each hotspot is clicked on is shown at 606, and the percentage of viewers that 
cHcked each hotspot is shown at 607. The percentage of viewers entering the hotspot- 
linked media via mouse is shown at 608, and the percentage of users entering the 
20 hotspot-linked media by clicking the mouse is shown at 609. Also, the average time 
each hotspot is viewed by a user is shown at 610. 

Overlay data is shown at 61 1, indicating the overlay for each listed hotspot. 
Overlays are used to mark hotspots in the media presentations, and may comprise an 
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image that appears on or near a hotspot, a cursor that changes when passed over a 
hotspot, or no marking at all. Other overlays are possible, such as altering the contrast 
or brightness of a hotspot or marking the hotspot in other ways, and are within the 
scope of overlays for purposes of this application. 
5 Figure 7 illustrates a special version of a hotspot media report comprising data 

only on media that is advertising media. The advertising image name and size are 
shown at 701, and the chck-through URL the hotspot link links to is shown at 702. 
The number of times each advertising image was retrieved within a user session is 
shown at 703, with the number of times each image was retrieved by a unique user 

10 shown in parenthesis. The number of sessions in which the hotspot was cHcked is 
shown at 704, and the number of unique user clicks is shown in parenthesis. At 705, 
the click- through percentage shows the percentage of user sessions that result in a 
click-through, along with the percentage of click- throughs by unique user in 
parenthesis. The average time spent viewing the advertising image is shown at 706, 

1 5 and the hotspot name is shown at 707. The media name of the media that contains the 
ad is shown at 708, which is typically video. Finally, the URL of the advertisement on 
the ad server is shown at 709. In altemate embodiments of the invention, advertising 
pages may be displayed along with other media in a media report, and may be 
individually selected for display by user-configuration of the media report. 

20 Although specific embodiments have been illustrated and described herein, it 

will be appreciated by those of ordinary skill in the art that any arrangement which is 
calculated to achieve the same purpose may be substituted for the specific 
embodiments shown. This application is intended to cover any adaptations or 
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variations of the invention. It is intended that this invention be hmited only by the 
claims, and the full scope of equivalents thereof. 
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